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[ METHOD AND APPARATUS FOR 
ANALYZING A DESIGN] 

Background of Invention 

[0001] The present invention generally relates to a method and an apparatus for 

analyzing and/or optimizing a design and more particularly, to a probabilistic based 
software based system which may be used to selectively analyze the overall reliability, 
robustness, and other features and/or characteristics of a design which is created by 
and/or which is based upon a computer aided design type model, effective to allow 
for the production of an item having desirable features and characteristics. 

[0002] Traditional design techniques require the creation of one or more physical 

prototypes of an item or element which is to be produced. The prototypes are then 
subjected to a variety of tests. Further, various changes are made to the prototypes 
and/or new prototypes are created as a result of these tests and these changes or 
modifications are tested in order to allow for the production of a final product which 
has desirable characteristics and features (e.g., a relatively high reliability). 

[0003] While these traditional design techniques or methods are effective, they are 

relatively costly and time consuming. One attempt at improving upon these drawbacks 
of these traditional design techniques utilizes a computer generated or computer 
aided design "CAD" model which simulates the item or product which is to be created 
and which represents and/or comprises a certain design of the item or product. The 
computer system may be used to perform various tests and/or modifications upon the 
model or design, thereby allowing a user to determine the desirability of the design. 

[0004] while this approach does reduce the amount of time required for testing, it does 
not reliably assess the operation of the item or product in a "real operational 
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setting"since this approach does not account for variations or dynamic changes 
occurring in the values of the various variables used to model the item or product 
which commonly occur in a "real operational setting". This deterministic approach 
therefore does not allow a product or item to be analyzed in a "real world"situation 
and does not reliably allow for the production of an item having desired 
characteristics and/or attributes. 

[0005] There is therefore a need for a new and improved system which allows computer 
type models to be created of an item or product and which further allows these 
computer type models to be analyzed and tested in an environment which 
substantially simulates the "real operational environment" into which the produced 
item or product is to be operationally placed, thereby allowing for the creation of an 
item or product having a relatively high reliability, robustness and various other 
features and/or characteristics. 

Summary of Invention 

[0006] It is a first non-limiting advantage of the present invention to provide a method 
and an apparatus for analyzing a design based upon and/or represented by a 
computer generated model of an item or product in a manner which overcomes some 
or all of the previously delineated drawbacks of prior methods and apparatuses. 

[0007] It is a second non-limiting advantage of the present invention to provide a 
method an apparatus for probabilistically analyzing a design based upon and/or 
represented by a computer generated model of an item or product in a manner which 
overcomes some or all of the previously delineated drawbacks of prior methods and 
apparatuses. 

[0008] It is a third non-limiting advantage of the present invention to provide a method 
and an apparatus for analyzing a design based upon and/or represented by a 
computer generated model of a product or item in a cost effective and efficient 
manner, effective to allow an item or product to be produced having a relatively high 
reliability and various other desirable features and characteristics. 



[0009] 



According to a first aspect of the present invention, a system is provided to 
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analyze a design which is based upon a computer generated model. The system 
includes a computer that operates under stored program control and which 
probabilistically analyzes the computer generated model. 

[0010] According to a second aspect of the present invention, a method for analyzing a 
design which is based upon a computer generated model type design is provided. The 
method includes the steps of receiving the computer generated model; creating at 
least one variable; and probabilistically analyzing the computer generated model by 
the use of the at least one variable. 

[001 1 ] These and other features, aspects, and advantages of the present invention will 
become apparent from a reading of the detailed description of the preferred 
embodiment of the invention and by reference to the following drawings. 

Brief Description of Drawings 

[001 2] Figure 1 is a block diagram of a computer analysis system which is made in 
accordance with the teachings of the preferred embodiment of the invention in 
communicative combination with a conventional computer aided design system; 

[0013] Figure 2 is a flowchart illustrating the sequence of operational steps which 
comprise the methodology of the preferred embodiment of the invention; 

[0014] Figure 3 is a diagram, which is created in step 30 of the flowchart which is shown 
in Figure 2; 

[001 5] Figure 4 is a flowchart which illustrates a sequence of operational steps used to 
generate a performance surface as required by the methodology of the preferred 
embodiment of the invention; 

[0016] Figure 5 is a chart illustrating the response of a design which is analyzed by 

system 1 0 by the use of certain sampled design points in accordance with step 1 20 of 
the methodology of the preferred embodiment of the invention; 

[0017] 

Figure 6 is a flowchart illustrating a sequence of operational steps used by the 
methodology of the preferred embodiment of the invention to substantially optimize 
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the developed approximation model; 

[001 8] Figure 7 is a diagram illustrating the methodology used by and/or incorporated 
within the system of the preferred embodiment of the invention to determine a "most 
probable point" and 

[001 9] Figure 8 is a diagram illustrating the methodology used by and/or incorporated 

the system of the preferred embodiment of the invention to determine the robustness 
of a variable or parameter. 

Detailed Description 

[0020] Referring now to Figure 1 , there is shown a computer system 1 0 which is made in 
accordance with the teachings of the preferred embodiment of the invention and 
which selectively communicates with a conventional or typical computer aided design 
system 12. 

[0021] Particularly, computer system 10 includes a computer processor or controller 14 
which operates under stored program control, a display 1 6 which is operatively, 
physically, and communicatively coupled to the processor or controller 14 and which 
is adapted to visually display certain information which is received from the processor 
or controller 1 4 or from system 1 2 , and a keyboard 1 7 or other conventional and 
commercially available input device which is physically, operatively, and 
communicatively coupled to the processor or controller 14 and which is adapted to 
allow a user of the system 1 0 to command the system 1 0 to selectively perform one or 
more operations. 

[0022] Conventional computer aided design model 1 8 represents and/or comprises a 
certain design or product which may be selectively communicated to the system 1 0 
(e.g., to the controller or processor 1 4) from the system 12. This model (e.g. the 
design which is represented by the model 1 8) is then evaluated according to the 
methodology of the preferred embodiment of the invention. 

[0023] 

Referring now to Figure 2, there is shown a flowchart 20 which illustrates a 
sequence of operational steps which may be selectively performed by the system 1 0 
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(e.g., by the processor or the controller 1 4 which operates under stored program 
control). The initial step 30 of flowchart 20 requires the creation of an analytical 
reliability and robustness or parameter ("P") diagram 32 by the user of system 1 0. 

[0024] Particularly, diagram 32, as best shown in Figure 3, includes a first column 34, 
which is denoted as "parameter number". Each row 36 within column 34 uniquely 
identifies a parameter within the computer added design model 1 8 which is to be 
analyzed. A "parameter" may be defined as some measurable attribute or 
characteristic of the received design 1 8 and may have one or more constituent 
variables. Diagram 32 includes a second column 38, which is denoted as "parameter 
description" and an entry 39 in the column 38 describes the parameter resident within 
the same row 36 as the entry 39. Diagram 32 further includes a third column 40 
which is denoted as "nominal"and an entry 41 in a row 36 of column 40 denotes the 
nominal value of the parameter which is referenced in the same row 36. Diagram 32 
further includes a pair of columns 42 which are denoted as "design range" and which 
include a fourth and a fifth column 44, 46. The fourth column 44 is denoted as "lower 
bound"and the fifth column 46 is denoted as "upper bound". An entry 45, within 
column 44, specifies the lowest feasible or acceptable/desired value for the parameter 
which is referred to in the same row 36 as the entry 45. An entry 47, within column 
46, specifies the highest feasible or acceptable/desired value for the parameter which 
is referred to in the same row 36 as the entry 47. 

[0025] diagram 32 includes a sixth column 48 which is denoted as "variation". An 

entry 49, within the column 48, specifies the amount by which the nominal value, 
resident within the same row 36 as the entry 49, varies in the "real physical or 
operational environment"(e.g., within a vehicle). The diagram 32 further includes a 
seventh column 50 which is denoted as "parameter in model?". An entry 51 , within 
column 50, denotes whether the parameter, which resides within the same row 36 as 
the entry 51 , is included within the model 1 8. The diagram 32 includes an eighth 
column 52 and is denoted as "surrogate?". An entry 53 within the column 52 
delineates whether the parameter, which is referred to in the same row 36 as the entry 
53, is a surrogate. The term "surrogate," as should be appreciated by those of 
ordinary skill in the art, delineates a variable which may be the physical manifestation 
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of another variable (e.g., the variable of temperature may manifest itself in a variable 
length of a desired product and therefore the variable of length may be a surrogate 
for the variable of temperature). 

[0026] Diagram 32 includes a pair of columns 54 and this pair of columns are denoted as 
"sensitivity available?". The constituent columns 56, 58 of column pair 54 are 
respectively denoted as "RT'and "R2". An entry 57, within the column 56, denotes 
whether the sensitivity is available for the parameter resident within the same row 36 
as the entry 57 for a first response of the model or design 1 8 to an input. An entry 59, 
within the column 58, denotes whether the sensitivity is available for the parameter 
resident within the same row 36 as the entry 59 for a second response of the model 
design 1 8 to an input. 

[0027] Diagram 32 further includes an eleventh column 60, which is denoted as "remark". 
An entry 61 within the column 60 denotes or comprises any remarks that the user of 
system 1 0 desires to make with respect to the parameter resident within the same row 
36 as the entry 61 . 

[0028] Diagram 32 includes a section 62 which is delineated as "noise factor table"and 
which includes entries 63 which are representative of and/or which comprise a noise 
or uncontrolled variable associated with the overall design and which impacts the 
performance of the overall design. 

[0029] Diagram 32 further includes a section 64 which is denoted as "study goai"and 
which has three possible entries 66, 68, and 70 which are respectively denoted as 
"assessment", "parameter design", and "tolerance design". Entry 66 denotes the 
reliability/robustness assessment of the design made by the user of the system 1 0 
upon the completion of the methodology of the preferred embodiment of the 
invention, entry 68 denotes the desired values for each of the parameters upon the 
completion of the methodology of the preferred embodiment of the invention, and 
entry 70 denotes the amount of variance which is acceptable or desired in each of 
parameter design values. 

[0030] 

Step 30 requires that at least one of the entries 66, 68, and 70 be selected and 
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defined by a user of system 1 0. Diagram 32 further includes a section 72 having a 
first entry 74 which is entitled "system input"and which requires a description of the 
input signal(s) which is (are) applied to the model or design 1 8 by the controller or 
processor 14, and a second entry 76 which is denoted as "system responses" and 
which has multiple entries 78 which require a description of the respective responses 
which are expected after one or more inputs have been applied to the model or design 
1 8. Step 30 is completed upon the completion of the diagram 32 and this diagram 32 
may be used to ensure that all of the necessary parameters are evaluated by the 
system 10 and to compare the analytical results of the system 10 against the desired 
attributes or characteristics of each of the parameters, thereby allowing the analyzed 
model or design to be used to construct an item or product having desired 
characteristics or attributes. 

[0031] Step 30 is followed by step 80 in which the controller 14 is made to create and/or 
select a design region with which to sample the performance surface associated with 
and/or created by the design 1 8 as it is placed into a simulated operating 
environment by the controller or processor 1 4. 

[0032] ^ sho^d b e apparent to those of ordinary skill in the art, the relationship 

between all of the input variables or parameters and the output or performance may 
be thought of as or cooperatively forms a nonlinear response surface (e.g., each 
allowable and unique combination of input parameters produce output values which 
cooperatively form a performance surface). One non-limiting advantage of the 
invention is that a relatively small portion (e.g., the portion may be continuous or be 
formed from a plurality of discreet and discontinuous sample points) of the design 
space (e.g., the space formed from the interrelationship between or all of the 
allowable or possible combinations of input values of all of the parameters or 
variables) and a relatively small portion of the performance surface is used to reliably 
approximate the performance surface of the entire design , thereby reducing the 
overall cost associated with the creation and operation of the simulation and reducing 
the amount of time in which the simulation system must be operated. In the preferred 
embodiment, the samples are "spread out" through the entire design space and 
cooperatively form a true representation of the design space. Moreover, in the 
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preferred embodiment, each sample point provides or is related to a certain 
performance sample point on the performance surface. Thus, the overall design space 
creates a performance space. The steps required by this portion or step 80 of the 
methodology 20 are delineated in the flowchart 90 of Figure 4 and, as delineated 
above, seek to determine what portion of the overall design space is actually needed 
to reliably and desirably approximate the overall performance surface. 

[0033] As shown in flowchart 90 of Figure 4, the first step 92 of the portion or step 80 of 
the methodology 20 of the preferred embodiment of the invention requires that a 
relatively random sampling be made of the design space using a modified latin 
hypercube sampling technique. That is, a traditional latin hypercube sampling 
technique does not provide optimal spacing between sample points and therefore the 
obtained sample does not reliably represent or approximate the overall design or 
performance surface. In the preferred embodiment of the invention, as is further 
delineated below, the conventional Latin Hypercube sampling technique is 
heuristically combined with conventional "greedy"and "Tabu" methodologies to 
achieve a unique or modified Latin Hypercube algorithmic combination which allows 
for a substantially optimized approximation of the performance space. 

[0034] Thus, in step 94, a Tabu set l T'is created and is initially made to be empty. An 
entropy analysis is applied to the previously obtained random samples, in step 96, 
and the "best" solution (e.g., the solution having the lowest entropy) is placed into a 
set denoted as "m+". 

[0035] 

Step 98 follows step 96 and, in this step 98, a pair wise or "greedy" substitution is 
made of the previously obtained samples and, in step 100, an entropy analysis is 
performed an each pair wise substitution and each entropy is compared with the 
entropy of the sample currently within the set "m+". Step 102 follows step 100 and, in 
this step 102, only those samples having a certain entropy (e.g., having an entropy 
below some threshold which may be equal to the entropy of the sample placed in 
"m+") are placed in the set "T". Step 1 04 follows step 1 02 and, in this step 1 04, the 
controller or processor 1 4, determines whether the required number of iterations or 
time has elapsed or whether the entropy has not sufficiently improved during a certain 
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time or sampling interval and, based upon this analysis, proceeds to step 1 06 and 
terminates methodology 90, or proceeds to step 98 in which another pair wise 
substitution is made and placed into the set "Tonly if its entropy is lower than the 
entropy of the current sample which is resident within the set "T". 

[0036] At the conclusion of flowchart 90, controller or processor 1 4 has a set of design 
points (e.g., a certain portion of the design space) which cooperatively create and/or 
cooperatively form a certain portion of the performance space or surface (e.g., the 
resultant performance surface is cooperatively formed by the solutions or output 
values of each of these samples) and this performance surface may be used to 
extrapolate or approximate the remainder of the performance surface. 

[0037] Step 1 20 follows step 80 and, in this step 1 20, a computer based simulation is 

conducted by the processor or controller 1 4 in order to determine the response which 
is obtained from such an analysis. 

[0038] That is, as shown by chart 1 30 of Figure 5, a set of design sample points or values 
1 32 (e.g., each set includes each of the parameters 39 specified in a single row 36 of 
the chart 32) is communicated to the processor or controller 14 and applied to and/or 
incorporated within the design 1 8, thereby causing the design 1 8 to respond by 
outputting at least one value 1 34 for each set of design or input points/values 1 32. 
That is, the design 1 8 may have many sets of design points 1 32 which are 
sequentially communicated to it and each set of design points or parameters 1 32 
causes a respective output response 1 34 to be created. These sets of design 
parameters or points 1 32 and the respective responses 1 34 are noted in step 1 20 of 
methodology or flowchart 20. 



[0039] 



Step 1 40 follows step 1 20 and, in this step 1 40, the controller or processor 1 4 
utilizes the publicly available MARS algorithms or methodology which is disclosed in 
the paper entitled Multivariate Adaptive Regression Splines, which is authored by 
Jerome H. Friedman, which is published in the Annals of Statistics (1991), vol. 19, No. 
1 , 1-141, and which is fully and completely incorporated herein by reference. 
Particularly, the MARS methodology is applied on each row or set of design points or 
parameters 1 32 and causes the creation of a respective output value for each such 
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row or set of design points or parameters 1 32. The difference between the output 
value which is obtained from the design 1 8 for a row or set 1 32 and the output value 
obtained from the MARS methodology for the same row or set of parameters 1 32 is 
defined as the "residual". In this manner, a residual value is created for each row or 
of design points or values 132. 

[0040] Further, in step 1 40, the controller or processor 1 4 uses the residual value from 
each row 36 or set of design points 1 32 and uses the publicly available Kriging 
methodology or algorithms to create an additional output value for each row 1 32 
(e.g., the residual value for a row 1 32 or set of design points 1 32 is used, by the 
Kriging methodology, to generate an output value for that row or set of design points 
1 32). The Kriging methodology or algorithms are set forth, for example and without 
limitation, in the paper entitled Screening, Predicting, and Computer Experiments, 
which is written by William J. Welch et al., which is published in Technometric; vol. 34, 
No. 1 (February 1992) and which is fully and completely incorporated herein by 
reference. The respective MARS and Kriging output values, for each row or set of 
design points 1 32, are then added. An output value 1 34 should be substantially equal 
to the addition of the MARS and Kriging value for the row 1 32 which generated the 
value 1 34 , thereby validating the use of the combined MARS and Kriging simulation 
methodologies to simulate the performance surface. At the conclusion of this portion 
of step 140, the respective MARS and Kriging output values for each row or set of 
design points or parameters 132, which have been added, are respectively stored 
within the computer or processor 1 4 as an output value. 

A certain optimization methodology is then performed in step 140 in order to 
ensure that the set of design parameters or points, obtained in step 80, provides an 
approximation of the performance surface to a desired level of accuracy. This 
methodology is shown within flowchart 1 50 of Figure 6. 

Particularly, flowchart or methodology 1 50 begins with an initial step 1 52 in which 
the controller or processor 1 4 retrieves the information resident within the chart 1 30 
of Figure 5. This information is then used by the controller or processor 1 4 in 
combination with the Kriging methodology software to evaluate the correlation or the 
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amount of non-linearity for each of the utilized parameters. In this manner, the most 
"critical" parameters are identified (e.g., those parameters whose behavior or output 
values have a respective non-linear relationship to the input values and which are 
relatively difficult to approximate, especially with a relatively small sample), in step 
1 54, a certain number of additional samples are made of those parameters having a 
relatively high level of non-linearity and these samples are added to those included 
within the chart 1 30 (e.g., the additional samples 1 32 and their respective output 
values 1 34 are noted on the chart 1 30 and stored within the computer or processor 
1 4). In step 1 56, the modeling error is evaluated on the new matrix of design points 
(e.g. the respective output values associated with these new samples which are 
predicted by the combined MARS and Kriging methodologies and the actual respective 
output value generated by and/or from the design 18 for these new sample points are 
compared). The difference between these respective values for each set of new sample 
points is defined as the modeling error. In step 1 58 the modeling error is compared 
with a threshold value. If, in step 1 58, each such error is below the threshold value, 
the process is concluded. Otherwise, the two sets of design points or parameters (e.g., 
the previously obtained and new sample points) are combined in step 1 59 and steps 
1 52, 1 54, 1 56, and 158 are again completed. At the conclusion of step 1 50, sufficient 
design parameter samples are obtained in order to produce an approximation of the 
performance surface having a sufficient degree of accuracy. 

[0043] Step 200 follows step 140 and, in this step 200, the design parameters which 
approximate the performance surface are input to the processor or controller 1 4 in 
order to obtain certain information about the overall design by the use of a successive 
linear approximation method or by use of any other conventional approximation or 
simulation methodology. Step 201 follows step 200 and, in this step 201 , the user or 
the controller or processor determines whether the design is satisfactory. If the design 
is satisfactory, step 201 is followed by step 203 in which the controller or processor 
14 adopts the design as the "final design". Alternatively, step 201 is followed by step 
205 in which the controller and/or processor 1 4 conducts an optimization process or 
methodology. 

[0044] 

In step 205, for example and without limitation, the controller 1 4 creates a 
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probability distribution function, for each output value (which is associated with 
and/or is found from at least one variable), such as 1 34, and then calculates the 
distance between a fist certain percentile value, for example and without limitation a 
ten percentile value, and a second certain percentile value, for example and without 
limitation, the ninety percentile value, thereby assessing the amount of variance or 
robustness in each of the output values. A large amount of distance between these 
respective percentile points evidences a large amount of undesirable and respective 
variance, and the system 1 0 searches for values of these variables which reduces this 
distance and allows for the creation of a more robust product. 

[0045] That is, as shown best in graph 300 of Figure 8, controller or processor 1 4 further 
evaluates the relationship between the value of the performance (e.g. appearing on 
axis 304) and the probability of occurrence (appearing on axis 302) in order to 
determine the robustness of a certain design setting. Particularly, a relatively robust 
design setting produces a relatively short distance 310 between a first percentile 
point, such as and without limitation, the tenth percentile point 312 and a second 
percentile point, such as and without limitation, the ninetieth percentile point 314. 

[0046] The controller or processor 1 4, in this step 200, utilizes the saddle point method 
with second order approximation in order to compute a probability, which may be 
used by way of example without limitation to obtain the most probable point. 
Particularly, in the most preferred embodiment of the invention the most probable 
point is representative of the setting of compound noise within the system. It should 
be noted that the use of the most probable point of the performance surface allows a 
simulation to be "probabilistically" operated or analyzed (e.g., at the most probable 
point) in order to allow the simulation to provide "real world"or "real 
operational"results. 

[0047] 

Referring now to Figure 7, there is shown a graph 2 50 which illustrates the 
technique or methodology used by controller or processor 1 4 to locate or fix the most 
probable point which is defined, within the system 1 0, as the point 260 which lies on 
the border 251 of the acceptable and non-acceptable performance spaces which are 
respectively denoted as spaces 252 and 254. That is, the controller or processor 14 
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evaluates the performance space by use of the design parameters 256, 258 (e.g., each 
performance point 259 is a function of each parameter 256, 258) and determines 
which of the points 259 is most probable. In other non-limiting embodiments of the 
invention, the performance space may be comprised of many variables and 
parameters. As further shown, within Figure 7, in the preferred embodiment of the 
invention, the performance spaces 252 and 254 are normalized, thereby having the 
nominal value at the origin and the most probable point thereby occurs on the 
boundary 251 and has the smallest distance to the origin 262 of any other point on 
the boundary 251. The sensitivity around the point 260 determines, in part, the 
amount of influence of the variables, which are plotted on one or both of the axes 
256, 258. 

[0048] That is, in the most preferred embodiment of the invention, the "influence" of a 
variable is defined by the total or sum of the sensitivity and the amount of variability 
("noise") of the variable and, in the system of the preferred embodiment of the 
invention, the controller 14 analyzes the influence of each parameter and each 
constituent variable in order to display to the user those parameters and/or 
constituent variables, for each design point, which are most influential in the overall 
design, thereby allowing the user to select design points having other parameters 
which may be easier to control and which have less influence. Lastly, in the preferred 
embodiment of the invention, the contribution of compound noise is dynamically 
calculated for each design point, which is utilized, in order to obtain a more accurate 
overall result. That is, the calculated compound noise may then be utilized by the 
simulation methodology, in a conventional manner, to achieve a more accurate overall 
result. 

[0049] It is to be understood that the invention is not limited to the exact construction or 
method which has been delineated above, but that various changes and modifications 
may be made without departing from the spirit and the scope of the inventions as are 
more fully delineated in the following claims. It should be further appreciated the 
combined MARS and Kriging methodologies allow for the creation of accurate 
simulation by use of only a relatively small portion of the design space and the 
performance surface. 
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